Enabling Monolingual Translators: Post-Editing vs. Options

We carried out a study on monolingual translators with no knowledge of the source language, but aided by post-editing and the display of translation options. On Arabic-English and Chinese-English, using standard test data and current statistical machine translation systems, 10 monolingual translators were able to translate 35% of Arabic and 28% of Chinese sentences correctly on average, with some of the participants coming close to professional bilingual performance on some of the documents. While machine translation systems have advanced greatly over the last decade, nobody seriously expects human-level performance any time soon, except for very constraint settings. But are todays systems good enough to enable monolingual speakers of the target language without knowledge of the source language to generate correct translations? And what type of assistance from machine translation is most helpful for such translators? We carried out a study that involved monolingual translators who had no knowledge of Chinese and Arabic to translate documents from the NIST 20081 test sets, being assisted by statistical machine translation systems trained on data created under the GALE2 research program. Our study shows that monolingual translators were able to translate 35% of Arabic and 28% of Chinese sentences, under a strict standard of correctness that scored professional bilingual translations as 61% and 66% correct for Arabic and Chinese, respectively. We found also large variability among the participants and between the documents in the http://www.itl.nist.gov/iad/mig/tests/mt/ http://www.darpa.mil/ipto/programs/gale/gale.asp study, indicating the importance of general language skills and domain knowledge. The results suggest that a skilled monolingual translator can compete with a bilingual translator, when using todays machine translation systems.